Datasets and models for the paper Understanding the Effects of RLHF on LLM Generalisation and Diversity: https://arxiv.org/abs/2310.06452