I see in data sources you're using the original slim orca dataset, why not: https://huggingface.co/datasets/Open-Orca/SlimOrca-Dedup that has filtering & OAI RLHF removed? I assume it was an oversight but unsure if there was an intentional reason behind it.