RLHF
Expanded definition
RLHF typically has three stages: supervised fine-tuning, reward-model training on ranked completions, then a policy update (historically PPO) that increases reward while staying close to the base model. Preference methods such as DPO can skip an explicit RL loop. RLHF shapes helpfulness, refusals, and style; it does not guarantee factuality. Reward hacking and rater disagreement are first-class failure modes.
Related terms
Explore adjacent ideas in the knowledge graph.
RLHF FAQ
What is RLHF?
RLHF, reinforcement learning from human feedback, trains a model to prefer outputs that raters rank higher, usually via a reward model.
How is RLHF used in AI systems?
RLHF typically has three stages: supervised fine-tuning, reward-model training on ranked completions, then a policy update (historically PPO) that increases reward while staying close to the base model. Preference methods such as DPO can skip an explicit RL loop. RLHF shapes helpfulness, refusals, and style; it does not guarantee factuality.
Related
Comparisons, tools, and models that connect to this idea.