Bias Identification

Example Workflow

Below are the steps to measure the sequence-classification abilities of a given language model (e.g., Vicuna-13B-v1.3) on a supported dataset (e.g., stereoset). Since this codebase invokes vLLM for efficient generation via the OpenAI completion API, these steps should work for all models that vLLM supports, as well as all OpenAI models supporting the completion API endpoint.

Set up a vLLM "OpenAI-Compatible" server on a GPU server (instructions). Please open the sbatch_normal.sh script and modify line 51 to the desired model.

sbatch sbatch_normal.sh

You only need to run one GPU server. If you are running multiple requests you might need to request more GPUs by increasing -n (***UNTESTED - might need to modify the sbatch file).

If successful, you will see a message like Submitted batch job <id>. Look up this job with squeue -j <id>, and find the log file log_for_<id>_node_0.log

Next, create a .env file from .env.test. Fill in the name of the model and the URL of the GPU runtime (found with squeue -j <id>). For example, if the runtime is on gpu007 at port 8000, the API_BASE variable should be set to http://gpu007:8000/v1.

Note that MODEL_NAME should exactly match the value of the --model flag. For example, if vLLM is running on a local copy of the model, this variable should be set to the same path name, e.g. MODEL_NAME="/ssd005/projects/llm/vicuna-13b".

export API_BASE_REF="http://gpu027:8000/v1"
export API_BASE="http://gpu027:8000/v1"
export MODEL_NAME_REF="/model-weights/Llama-2-13b-chat-hf"
export MODEL_NAME="/model-weights/Llama-2-13b-chat-hf"
export DATASET_NAME="bbq"
export TEMPLATE="control"
# the code will not use ChatGPT but it still needs an API key to run
export API_KEY="sk-xxx"
export API_KEY_REF="sk-xxx"

To launch the generation client, run the following command on a CPU machine. Refer to parallel_generate.py for the full list of command line options. For example:

source .env.test && python3 -m evaluation.parallel_generate --tag=$(date -I)-llama2-13b-${DATASET_NAME}-${TEMPLATE} --num_examples=-1 --template=${TEMPLATE}   --dataset=${DATASET_NAME} --dataset_split="test"

This command will save results under data:

• Template json
• Streaming output jsonlines (updated on the fly in real time as the experiment runs)
• Output json

To generate metrics, visualizations, and cherry/lemon markdown files, run the following command on a CPU machine:

source .env.test && python3 -m evaluation.evaluate_bias --input_json=data/predictions-[tag from above]-streaming.jsonl --title=${DATASET} + ${TEMPLATE} --majority_vote=1 --context=bbq  --output_scale=8.0

To build contrastive datasets, start by running the get_metrics script to generate "lookup" json files. (See parsed_entry_lookup_output_path) After that, run the following command:

python3 -m evaluation.build_preference_dataset \
    --input_lookup_json=data/predictions-20230728c1-stereoset-vicuna-13b.json.lookup.json \
    --input_templates_json=data/predictions-20230731a1-vicuna-13b-stereoset-template.json \
    --output_dataset_path="data/artifacts/preference-stereoset-vicuna-13b-20230728c1"

Adding a new dataset

Please help keep the pipeline in this repo as dataset-agnostic as possible. To add a new dataset, all what you need to change are the following files:

• Add dataset loading and formatting logic to dataset_utils.py; register the dataset in the dataset_loaders dictionary.
• (Optional) Add new templates to templates.py.
• (Optional) Add new "context" (name of choices and labels, etc.) to get_metrics.py.

Run the following unit test to ensure that your dataset is registered properly:

export DATASET_NAME=NEW_DATASET_NAME  # e.g., "stereoset"
export DATASET_SPLIT=EXAMPLE_SPLIT  # e.g., "validation"
python3 -m unittest evaluation.tests.data_tests.DatasetLoaderTests

Adding a new template

For new prompt templates, add them to the dictionary in evaluation/templates.py. When running the script, reference the key name that you have given to the new template.

Running KG Experiments

To run the KG experiments, run:

source .env.llama13b && python3 -m evaluation.parallel_generate --tag=llama2-13b-bbq-adv-explanation-0.5 --num_examples=-1 --template="kg" --explanation=True --setting="adv" --dataset=bbq --temperature 0.5;

Alternatively, you can look at batch_experiments.sh for more example commands.

Running CoU and ES experiments

To run these experiments, please follow the steps above for modifying .env.llama13b. Then, run batch_cou_es.sh